Arquillian (1.0.2) runs the TestNG lifecycle for every test!

This is critical to understand. It means that BeforeXXX/AfterXXX methods are all called for every test. Effectively this means that the only methods you can effectively use are BeforeMethod and AfterMethod.

Know that BeforeMethod and AfterMethod are actually invoked twice per test.

Arquillian has two modes of testing, client and in-container. Client testing is such that the tests are basically remote clients and hit the deployed server in that fashion. We use the other mode, which is to run the tests in the container, meaning we can inject InitialContext, EntityManager, etc.. into the tests. Due to architectural integration issues, Arquillian calls BeforeMethod twice, once for each mode. We need to execute our code only for the in-container invocation.

The preferred mechanism for a BeforeMethod or AfterMethod is now to extend the base class methods offered in AbstractEJB3Test. So:

// NOT recommended
@BeforeMethod
public void myBeforeMethodName();

// Recommended
// By overriding the method you'll get built in in-container invocation only, as well as standard error handling.  Note that there are a few flavors
// of the method, some taking parameters.
// IMPORTANT: use protected, not public.  Public methods are considered test methods by testng if the class is annotated with @Test.
@Override
protected void beforeMethod();

// If you must use the annotation ensure you protect the invocation
@BeforeMethod( some attributes you just have to have )
public void myBeforeMethodName() {
  if ( !inContainer()) {
    return;
  }
  ...
}

Handling test resource files

Remember, the tests are no longer running with a current working directory of the maven module directory. The tests are being deployed to and run inside the AS7 server. This impacts the way test resources are handled, and also how to interact with the file system.

• Add test resources to git repo AND the deployment in AbstractEJB3Test
  The itests-2/src/test/resources directory has all of the static files we use with the tests. These files must now be included in our rhq.ear test deployment. To do that, you must add it to the deployment that is built up in AbstractEJB3Test. Look in there for examples.

• Use the default temp directory as opposed to CWD
  Many tests assumed the current working directory and performed file system work in "." or "target", etc. From now on the convention is to use the temp directory:

  // do filesystem work under /tmpdir/testclass/xxx
  new File(System.getProperty("java.io.tmpdir"), this.getClass().getName() + "/whateverSubDirs);

Know that we use an extension for running several test classes against a single deployment

By default Arquillian with testng will:

1. Start container

2. Deploy test deployment

3. Undeploy test deployment

4. Shutdown container

Although good for test exclusivity, it is prohibitively slow for a project with many test classes that doesn't need that level of test isolation. Working with the Arquillian developers we've put in place an Arquillian extension that allows us a single container start and deploy. This is in class SuiteDeploymentArquillianExtension.java which is based off of https://gist.github.com/3975179.

It should be noted that this extension uses Arquillian's (not testng) BeforeClass/AfterClass hooks, so those become unavailable for other use, but that is not an issue for us as we don't have any other extensions and observing those (or any other) events.